How do quantitative studies involving people with dementia report experiences of standardised data collection? A narrative synthesis of NIHR published studies

Background People with dementia are routinely included as research participants in trials and other quantitative studies in which they are invited to respond to standardised measures. This paper reviews the reporting of standardised data collection from people with dementia in reports published in the National Institute for Health and Care Research (NIHR) Journals Library. The aim was to understand how the administration of standardised, self-report measures with people with dementia is reported in NIHR monographs and what could be learnt from this about the feasibility and acceptability of data collection approaches for future studies. Methods This was a systematic review with narrative synthesis. Broad search terms (Dementia OR Alzheimer*) were used to search the NIHR Journals Library website in December 2021. All studies that used (or intended to use) standardised measures to collect research data directly from people with dementia were eligible for inclusion. Information was extracted (where reported) on the process of data collection, dementia severity, levels of missing data and the experiences and reflections of those involved. Results Searches returned 42 records, from which 17 reports were assessed as eligible for inclusion, containing 22 studies. Response rates from participants with dementia in these studies varied considerably and appeared to be related to dementia severity and place of residence. Little information was reported on the process of data collection or the reasons for missing data, and most studies did not report the experiences of participants or those administering the measures. However, there was an indication from two studies that standardised data collection could provoke emotional distress in some participants with dementia. Conclusions Through this review we identified both variation in levels of missing data and gaps in reporting which make it difficult to ascertain the reasons for this variation. We also identified potential risks to the well-being of participants with dementia which may be associated with the content of standardised measures and the context of data collection. Open reporting of and reflection upon data collection processes and the experiences of people involved is essential to ensure both the success of future data collection and the wellbeing of study participants. Trial registration Registered with Research on Research https://ror-hub.org/study/2905/. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-024-02148-y.


Background
People living with dementia make up a significant proportion of the adult population using health and social care services [1] yet historically this group could be excluded from research participation [2].Over the past two decades, a growing literature has both argued the importance of involving people with dementia as participants in research [3][4][5][6] and given practical advice about the best ways to achieve this [7][8][9][10].However, the vast majority of good practice literature focuses on qualitative methods, emphasising the importance of flexibility and foregrounding the voice of the person with dementia [11][12][13], whilst relatively little has been written about the practice of involving people with dementia as participants in trials or other quantitative research [14,15].Despite this, standardised measures have been developed to collect quantitative data specifically from this group [16][17][18][19][20] and a number of existing measures have been validated for use with people with dementia [21].Detailed monographs set out the development and psychometric properties of these measures, and some come with scripted instructions for their administration [18,22], but very little has been published examining the process of data collection or the experiences of the people involved.
In the absence of an abundant literature on good practice in quantitative research with people with dementia, this paper reviews the reporting of data collection in published National Institute for Health and Care Research (NIHR) reports where standardised measures were used with people with dementia for research purposes.The review was conducted as part of a doctoral research project aiming to better understand the process and experience of structured data collection in a large study of people with dementia (the DETER-MIND programme) [23].The overall research explores what factors influence the answers given by people living with dementia to standardised measures, how these might change over time as dementia symptoms progress, and what the implications are for research incorporating standardised measures and the people involved.The aim of the review was to consider how the administration of standardised, self-report measures with people with dementia is reported in NIHR monographs and what can be learnt from this about the feasibility and acceptability of data collection approaches for future studies.A greater focus on acceptability in quantitative dementia research should be of interest to trial and other quantitative researchers, and to all those interested in the ethics of dementia research.Key debates in dementia trials research ethics have tended to focus on capacity, consent and use of proxy data [24] but there may also be ethical considerations related to the experience of research participation that are as yet unidentified.

Standardised self-report measures for people with dementia
Questionnaires used in trials and cohort studies to measure outcomes, or assess health or psychosocial traits, are often standardised (with set wording, ordering of questions and answer scales) in order to ensure different scores reflect true differences between participants or time points rather than variation in the ways questions were asked [25].When measures are administered faceto face, it is expected that interviewers will introduce and read each question to participants in the same way and instruct them to provide an answer in the required format in order to minimise the chances of interviewer bias [26,27].Since the early 2000s, research has indicated that people with dementia can (and should be enabled to) respond to such measures to appraise their own health and quality of life in this standardised way for research purposes [28][29][30].A number of dementia specific measures have been developed; the most commonly used in published health research are DEMQOL [18] and QOL-AD [17].These and other similar measures are referred to in this paper as 'self-report' to distinguish them from informant (family or professionals') ratings of the person's quality of life, proxy questionnaires (which typically ask family carers or professionals to consider how they think the person with dementia would score their own quality of life [31]) or observational measures such as the QuIS [32].
A number of reviews of the relative merits of different dementia specific and generic measures of quality of life have been published, but most tend to compare only the psychometric properties of measures, and although some do report rates of missing data and 'feasibility' , there is rarely any mention of participant experience or focus on ability to respond to the items contained in the measures [33][34][35][36].Whilst acceptability and respondent burden are important attributes of any measure [37] these tend not to be examined in the literature to the same degree as validity and reliability [38] even in dementia research where cognitive impairment and altered emotions may make this particularly relevant [12,39].Definitions of acceptability vary but tend to cover the degree to which participants find a measure difficult or distressing to complete, indicators for which can include refusal rates, response rates and administration time.As Fitzpatrick et al. note [38]: 'Pragmatically, trialists using patient-based outcome measures are concerned with the end result; whether they obtain as complete data from patients as possible… However, we need to consider the different components of acceptability in turn to identify sources of missing data.' (p40) Krestar et al. [40] did examine people with dementia's ability to respond to different types of structured questions, concluding that participants with greater cognitive impairment struggled more when presented with bidirectional response categories (which contain two distinct concepts -such as 'strongly disagree, disagree, agree, or strongly agree') than when presented with scales which varied along only one dimension (such as "not at all, just a little, a fair amount, or a great deal").Those who struggled were permitted to use simpler, dichotomous (yes/know) response categories, but most standardised measures do not allow this option.More recently, Cohen et al. [41] found a relationship between participants' self-reported cognitive abilities and response times to standardised questions, with those with greater self-reported cognitive impairment taking longer to respond to questions with more syllables, those which contained abstract concepts, and those which required a degree of evaluation (as opposed to simple recall of frequency, for example).However, participants were recruited because they had one of five long-term neurological conditions, which may be accompanied by dementia, but acceptability for people with dementia in particular was not the primary focus of that study.

The review
This paper presents a narrative synthesis of the reporting of standardised data collection from people with dementia in reports published in the English National Institute for Health and Care Research (NIHR) Journals Library [42].The aim of the review was to explore the use of standardised, self-report measures with people with dementia as reported in the published monographs of research funded by the NIHR and available on the NIHR Journals Library website, focussing on the following indicators of experience, feasibility and acceptability: • The level of missing data, in terms of both response rates to full measures and item completeness within individual measures, where this was reported • The process of measure administration (how measures were used with people with dementia) and any reflections upon this process • The views and experiences of the people involved, including: • Participants with dementia and their carers or other supporters • The research team, including the report authors and the researchers collecting data • The impact of dementia severity on the experience of, and response rates for, standardised measures

Methods
This paper presents a narrative synthesis of NIHR funded dementia research.A narrative synthesis is 'an approach to the systematic review and synthesis of findings from multiple studies that relies primarily on the use of words and text to summarise and explain the findings of the synthesis.'( [43], p5).We conducted systematic searches, selection, and data extraction to ensure comprehensive coverage (within tight boundaries) but approached the collation and presentation of findings narratively to allow for clarification and insight.The decision to focus on NIHR funded research reports was made for two reasons.Firstly, the NIHR is internationally renowned as a leader in public and patient involvement in research, so it would be reasonable to expect that studies funded by this body would exhibit good practice in data collection involving potentially vulnerable participants and those with additional communication needs.Secondly, a number of NIHR funding streams require the research to be published in detailed monographs adhering to strict guidelines which typically run to 50,000 words.These reports contain full details of study methods, as well as study findings and limitations and thus offer sufficient space to detail any observations or learning about the use of study measures and the experiences of people involved.
All dementia focussed research reports published on the NIHR Journals Library website [42] that reported the use of standardised self-report measures with people with dementia for research purposes were targeted for review.Here 'self-report' does not necessarily mean that participants responded to a question or measure independently (for example, online or on a paper questionnaire), indeed it is more common for older people and people with dementia to be asked to answer questions verbally in a structured face-to-face interview [26].Thus, the term 'self-report' here means specifically that questions were expected to be answered directly by the person with dementia rather than by a proxy or informant, and scores were not based primarily on the ratings or judgement of another person.

Search scope and dates
All final reports of studies listed in the NIHR Journals Library [42] involving standardised self-report data collection from people with dementia were in scope.Final searches were conducted on 17 th December 2021 with no restrictions on date of publication.The journals library was established in 1997, initially only covering the journal Health Technology Assessment, but by the date the searches were conducted the library comprised five NIHR open-access journals.

Search terms and screening
Broad search terms (Dementia OR Alzheimer*) were selected in order to ensure that all potentially relevant reports were identified.No other search terms were used.Abstracts and (where the abstracts were not sufficiently clear) the full texts of all returned records were screened for eligibility.

Inclusion criteria
• The study used (or intended to use) standardised selfreport measures to collect research data from people with dementia

Exclusion criteria
• No standardised self-report measures were used, or intended to be used, with people with dementia Where a report included multiple studies, one or more of which might meet the criteria, each individual study was screened for eligibility.Where a study included standardised data-collection from a subset of people with dementia, this was included, so long as at least some of the data were to be self-reported by participants with dementia themselves.

Data extraction
Data were extracted from all included reports into an Excel spreadsheet under the following headings: • Element of study involving standardised data collection from people with dementia As this review formed part of a PhD study, the first author worked independently to select and review studies, with regular supervision by co-authors (YB and KB).After KG had completed data extraction, KB read and independently extracted data from two of the studies to cross check the data.

Results
A search of the NIHR Journals Library database using the terms Dementia OR Alzheimer* conducted on 17th December 2021 returned 42 reports out of a possible 2027.Figure 1 shows a PRISMA flow diagram with numbers of reports included and excluded, and reasons for exclusion.As some of the included reports contained more than one eligible study (for example, reports of programme grants), more studies were included (n = 22) than the total number of selected reports (n = 17).
Table 1 gives a full list of all self-report measures used with people with dementia in the 22 included studies and the primary outcome measure (where applicable).Some studies restricted participation to people with mild to moderate dementia, whereas others included people with all stages of dementia (including those with more severe symptoms).We found it useful to group studies that included participants with a similar level of dementia severity together, to enable response rates to be viewed in light of the mix of people involved.Table 1 groups studies under two headings: • Studies collecting data from people with mild to moderate dementia only • Studies collecting data from people with all stages of dementia Eight of the studies -in seven reports [44][45][46][47][48][49][50] -collected data from people with mild to moderate dementia (based on professional/carer assessment or scoring on a standardised self-report measure like SMMSE).The remaining fourteen studies -in 12 reports [45,49,[51][52][53][54][55][56][57][58][59][60] -collected data from participants with all stages of dementia, including those with more severe symptoms.Some reports explicitly stated that dementia severity was assessed at baseline (and showed changes over time) whereas others reported severity as a static quality of the sample.A wide range of measurement tools was used across the included studies, most commonly to measure quality of life, cognition and various psychological characteristics.(These tend to be referred to by acronyms, so a glossary of measures is included as a supplementary file to aid comprehension.)Eight of the studies (in 7 reports: [44, 47-50, 54, 57]) had a self-report measure, to be completed by participants with dementia, as a primary outcome measure (usually alongside other self-report and carer or professional rated measures); seven studies (in 6 reports: [45,46,52,54,58,60]) had a carer or professional rated measure as the primary outcome measure; two used 'objective' measures such as eye examinations or brain scans as the primary outcome [47,53]; and five studies (in four reports: [51,55,56,59]) did not identify a primary outcome.If response rates were not explicitly reported, we calculated these (where possible) from data provided in tables and accompanying text in the reports.Some reports amended follow-up sample sizes to reflect withdrawals, resulting in response rates appearing higher at follow-up than in studies employing an intention to treat approach.Sample size (N) at each time point has been included in Table 1 (if this information was clearly available from reports).

Response rates, measure completeness and dementia progression
Table 1 illustrates that response rates (that is, the proportion of participants to complete each measure at each timepoint) varied considerably between studies, even where studies had similar designs.Studies with participants assessed as having mild to moderate dementia generally reported response rates of over 90% at baseline, but the degree to which this was maintained at follow-up varied (where reported).Response rates for studies that included people with more severe dementia varied more widely at baseline, from 20.1% (for DEMQOL) in a longitudinal study of a toolkit for incontinence [56], to 100% (for all baseline measures) in a feasibility study of a falls intervention [51].Overall, studies which included people with all stages of dementia were less likely to report high response rates at any time point than studies which restricted participation to those with mild to moderate symptoms.
Response rates, or information from which a response rate could be calculated, were not always reported clearly by measure and time point [45,49,50,52,57,60].In an observational study of dementia home

All stages eligible
The majority had mild to moderate dementia at baseline (94% had CRD scores between 0.5 and 2, the remaining 6% had a CDR of 3).
Self support for people with 'later stage dementia' [45], for example, the report states that 389 out of the 518 participants (75.1%) were interviewed at both baseline and 6 month follow-up, but it is not clear what proportion of each self-reported measure this group responded to at each time point.This is important as it does not necessarily follow that all 389 participants interviewed at both time points responded to all three of the selfreport measures each time.We know from other studies that response rates to different measures can differ even within a time point.In a study of life story work with people with dementia in care homes, for example, 64% of participants responded to QOL-AD at baseline, but only 31% of those same participants responded to DEMQOL at baseline [55].
Response rates appeared to be associated with setting (i.e., whether participants were recruited from community or residential care or inpatient settings), but it is difficult to separate this from dementia severity, which in theory could be higher (reflecting the need for residential care) but in practice was not always measured.One study, for example, abandoned a measure (the IDEA questionnaire) after it transpired it was 'too cognitively complex' ([57], p35) for most participants with dementia to respond to.The dementia severity of participants in this study (all of whom were recruited from inpatient or residential care settings) is not known because only 13 participants out of a sample of 332 completed the cognitive test.Similarly, dementia severity was not formally assessed in two studies by Gridley et al. [55] which recruited from inpatient and residential care settings and had response rates ranging from 0% to 64%.Another study ([58], Study 2) which recruited people with dementia from residential care home settings had so much missing data that planned imputation was not conducted.People with all stages of dementia were included in this study, but nearly half (49.2%) were assessed as having severe dementia at baseline.Response rates were higher in the study by the same team which only recruited participants still living in the community.Whilst this latter study also included people with all stages of dementia, less than 9% of the community cohort had severe dementia at baseline.
Another large study which recruited exclusively from residential care settings [60] excluded all data collected directly from people with dementia from the analysis because of high levels of missing data, as the authors explained in their limitations section: 'Owing to the variability in the ability of care home residents with dementia to self-report on measures of BSC and QoL, the primary and secondary analyses were conducted using staff proxy-completed measures' (p97).
By contrast, a study by Gathercole et al. [54] included people with all stages of dementia living in the community (not residential or inpatient settings), and reported higher response rates than the above studies, but lower than other studies which recruited from the community but restricted participation to people with mild to moderate dementia.
An RCT of individual cognitive stimulation therapy [48] reported very high response rates for multiple measures (typically close to 100%) which reduced only slightly over the 26-week follow-up period.In common with other large studies with high response rates, participants had mild to moderate dementia and were living in the community at baseline, and only people with capacity to consent and 'no major co-morbidities affecting participation' were eligible to take part.The authors note that tight eligibility criteria did restrict participation: 'In total, 1340 people were considered for recruitment to the study.From these, 356 were randomised and together constituted the final sample for the study.…Losses in 22% of cases were attributable to people with dementia not meeting the clinical criteria, indicating that this factor was, to some extent, a barrier to study recruitment.'

(p 5)
An implementation study of group based maintenance cognitive stimulation therapy (MCST) had similarly tight eligibility criteria [49], excluding people with severe dementia or any additional communication, physical or intellectual impairments, specifying that participants must 'have the ability to complete a cognitive and quality-of-life measure at three intervals over 1 year' (p.51).This study applied intention to treat analysis, using all available information provided by participants with dementia at follow-up regardless of whether they completed the intervention programme, but it is not clear whether the reduction in available data over time (they reported QOL-AD for 89 participants at baseline, 62 participants at first follow-up and 56 participants at second follow-up) was the result of withdrawal from the study, other loss to follow-up or some participants declining (or finding it difficult) to respond to the measures.Response rates by measure for the other two eligible studies in this programme (an RCT of MCST and an RCT of a carer supporter programme and reminiscence intervention) were not clearly reported.
It was very rare for studies to report measure completeness, that is, what proportion of the items in individual measures were completed by participants.Allan et al. [51] did note that 'All self-reported and proxy EQ-5D-5L questionnaires that were completed had no missing data for any of the domains.' (p89) but this level of detail was very much the exception and perhaps only reported because theirs was a small feasibility study.Statements such as the following were more common, where authors set out how missing data were handled, without specifying how much there was or from which measures: 'Complete-case data analysis was used initially to establish the results, followed by the analysis with imputations.When individual data points were missing within a scale, data were imputed by using scale/subscale means according to the validated rules for the measures.When an outcome measure total score was missing, it was imputed using a multiple imputation regression model ... ' ( [49], p29) Without clear information about response rates by measure, measure completion, and the reasons behind missing data, it is difficult to compare approaches to data collection in different studies or ascertain possible explanations for problems encountered.

The process of measure administration
Table 2 presents information reported from included studies on the process of measure administration: this included the views of participants on their experiences of taking part in the research (for example, from embedded process evaluations); and reflections by study teams on the data collection process or measures used.Ten studies (in nine reports) collected data from people with dementia face to face in participant's own homes or another place convenient to them [44, 45, 47-51, 53, 58], six studies (in five reports) collected data from participants in care homes or hospital settings [55,[57][58][59][60], and the remaining six studies (in five reports) did not state where data were collected [46,49,52,54,56].
Overall very little information was given about the circumstances or activities that took place during data collection encounters.Typically, reports featured a statement such as 'outcomes were obtained during a face-to-face assessment by a researcher….' ( [47], p15).Occasionally, a little more detail was offered, as in this example:

'The questionnaire measures were arranged into booklets, which facilitated their ease of delivery during the interviews. If a participant became tired, or if it was requested by participants or deemed appropriate by the researcher, an interview was occasionally broken off part-way through and then continued on another day. ' ([58], Study 4, p120)
Orgeta et al. and Woods et al. [48,50] used very similar wording to explain that assessors occasionally arranged to return 'to complete assessments where an interviewee became tired, or where it was otherwise requested by participants or deemed appropriate by the assessor' ([50], p14).No further information was given in these reports about how often participants requested that an interview be paused and completed later, or why this might be 'deemed appropriate' by the researcher/assessor.The participants in these two latter studies had mild to moderate dementia.While a number of other studies used more measures and/or included participants with more severe dementia, they made no reference to breaking data collection sessions into more manageable chunks.It is unclear here whether such adjustments were not made, or just not reported.
Most studies did not report intervening to improve the accessibility or acceptability of data collection tools or processes for participants with dementia, other than to collect data face to face at a location acceptable to participants and employing trained research workers.Allan et al. [51] did reduce the number of questions in their health utilisation questionnaire and Orgeta et al. [48] used show cards (which typically present the answer scales visually) to support people with dementia to respond to the measures, with accompanying reports of high response rates.Surr et al. [60], on the other hand, used an adapted version of QOL-AD developed specifically for use in care homes which has 'simple language' and a four-response answer scale that is consistent across all questions, but still did not collect enough data directly from residents with dementia to enable their data to be used in the analysis.Similarly, Kinderman et al. [57] reported that people with dementia on hospital wards and in care homes received 'assistance from skilled clinicians' (p53) to answer QOL-AD, but most participants still did not complete this measure.

The views of participants and reflections of study teams
Occasionally a report included a few lines about why participants did not attempt to complete assessments.Gathercole et al. [54] for instance note 'this could have been for several reasons, including disagreement with allocation, burden of assessments and delays in assessments being completed.' (p40) Unusually, the Bowen et al. [53] report sets out in some detail the reasons for missing scores for a specific measure (SMMSE) for 54 participants: 'These participants mainly comprised those for whom no coherent responses were obtained when attempting the test, and so could not be assessed using the SMMSE, and a small number who were unavailable, asleep or uncooperative on the day of recruitment, and so the test was not carried out.' (p37) 'The measures chosen were all designed to be completed by people with dementia but response rates leading to usable data were low and varied between the measures … The main reason for participants not completing a measure was that they were not able to understand and/or respond to the questions.' (p69) [55] Gridley (2016) Feasibility study of life story work with people with dementia (mental health inpatient assessment units) Face to face data collection in inpatient units.Two researchers in the project team collected data at all time points.
One participant with dementia residing in a mental health inpatient assessment unit was interviewed for the process evaluation (see above) As above [46] Howard (2020) RCT of Minocycline for mild Alzheimer's Disease

Not reported
Authors suggest that the low completion of SMMSE at follow-up was due to people on the higher dose withdrawing from the treatment and: 'Although the trial protocol specified that outcome assessments should be obtained irrespective of treatment compliance, this could not always be achieved despite the vigorous efforts of the trial team.' (p22) [56] Iliffe (2015) EVIDEM-C: mixed-method longitudinal study looking at incontinence

No information
A feasibility study included interviews of carer participants about their experiences of data collection, but people with dementia were not interviewed.
There is no reflection on the small numbers of completed self-report measures in the discussion or limitations sections.
[ In most care homes, two researchers interviewed residents and care staff concurrently in separate rooms.In some instances, additional visits were arranged to complete interviews if, for example, participants became tired.
Not reported.Process evaluation included data from interviews with care home staff and a focus group with 'stakeholders' (not including anyone with dementia).The focus of both was the implementation of the intervention, not data collection processes.A section of the report focusses on missing data but says little about the causes of missing data other than 'The researchers endeavoured to collect as many data as they could.However, two types of missing data were inevitable: missing items within a measure and missing time points.Missing items were attributable to researcher error or participants declining to answer individual questions.When questionnaires had recommended rules for managing such missing items, these were applied.' (pg 40) [58] Moniz-Cook (2017) Observational cohort study of people with dementia and challenging behaviour living at home and their carers (Study 4) All interviews were conducted in the person's home unless they requested an alternative location.Occasionally interviews were broken into chunks (either at the participant's request or if the researcher deemed it appropriate) Not reported.One person with dementia attended the stakeholder consultations (1/39 participants).The focus of discussion was the intervention and wider access to services.No mention of study methods or experiences of data collection.As in study 2, the authors comment on the discrepancy between self-report and proxy QoL measures but do not reflect on the implications of this or low self-report response rates in the results, discussion, limitations or conclusion sections.
[59] O'Brien et al. (2021) Pilot RCT of a management toolkit for Lewy Body Dementia The setting for the study was secondary care memory assessment and movement disorder services in England.All assessments were undertaken by members of the National Institute for Health Clinical Research Network As part of an embedded qualitative study it is noted that ' … patients and carers highlighted some issues with question wording, typically with the same questions identified as problematic by clinicians' (p39).No further information about the nature of the feedback or which questions were referred to.In the limitations section the authors note: 'There were occasionally some tensions between research paradigms, in particular in relation to managing qualitative feedback on question wording in the assessment toolkits, with the value given to 'validated' questions derived from clinical research.' (p44).No further explanation is given.

Table 2 (continued)
[48] Orgeta (2015) RCT of individual cognitive stimulation therapy for dementia All research activities took place in participants' homes.Showcards were used to support participants with dementia to respond to the measures.If participants felt uncomfortable with the assessment this was discontinued (and rescheduled where appropriate).
An embedded qualitative study explored experiences of 22 participants with dementia, but the focus was the intervention.No mention of participants' experiences of data collection.
The authors reflect that the clinical criteria for inclusion of people with dementia were a barrier to recruitment.
[49] Orrell (2017) RCT of Maintenance Cognitive Stimulation Therapy (MCST) No information (page 25 explains that: 'Half of the sample was recruited from care homes and half was recruited from community settings' , but the report does not specify the context in which data collection took place).

Not reported
Commented that, in their analysis, 'DEMQOL seemed to be a more sensitive instrument than the QOL-AD for measuring change in quality of life in dementia.' Alternatively, 'the two measures may be measuring different aspects of quality of life.' (p37) The authors called for more research to explore the differences between these two measures further.
[49] Orrell (2017) Maintenance Cognitive Stimulation Therapy implementation study (observational) Interviews with people with dementia were carried out by a researcher or staff member who was trained to undertake the assessment and had training in Good Clinical Practice and taking informed consent.
Three focus groups were conducted with 10 people with dementia and 5 staff members looking at the experience and effect of maintenance cognitive stimulation therapy.No mention of participants' experiences of data collection.No reflection on data collection, response rates or experiences of participants with dementia providing data.
[49] Orrell (2017) RCT of a Carer Supporter Programme and reminiscence intervention Face to face interviews were held at 'times and venues organised to accommodate the carer' s needs and preferences.' (p78) The questionnaire for the person with dementia was always completed with the researcher.Not reported.Participants are generally referred to as carers, although self-report data were collected from people with dementia.No reflection on data collection, response rates or experiences of participants with dementia providing data.
[60] Surr (2020) Cluster RCT of dementia care mapping in residential care settings (with a cross sectional element added at 16 months) The research took place in care homes.Little information is given about the data collection context other than to note that data were collected by 'researcher interview' Process evaluation focussed on implementation of the intervention, not data collection Response rate was recognised as a problem and data collected directly from people with dementia was excluded from the analysis.This was noted as a limitation of the study: 'Owing to the variability in the ability of care home residents with dementia to self-report on measures of BSC and QoL, the primary and secondary analyses were conducted using staff proxy-completed measures' [50] Woods (2012) RCT of group reminiscence for people with dementia and carers Face to face interviews were conducted in participants' homes.
Measures were arranged in a number of booklets.'A second visit was sometimes made to complete assessments where an interviewee became tired, or where it was otherwise requested by participants or deemed appropriate by the assessor.' (p14) No process evaluation, and this is identified as a limitation in the discussion From the embedded study of EQ-5D the authors concluded: 'Participants with dementia were able to complete the EQ-5D in a face-to-face interview, in line with evidence on suitability of this healthrelated quality-of-life instrument in this patient group.' (p49) rates achieved to the length of the measure, suggesting that in future more attention should be paid to the tradeoff between the value of potential data from a measure, and the likelihood of obtaining enough data to be valuable: '…the ADAS-Cog is often used in clinical trials because it can determine incremental improvements or declines in cognitive functioning.Despite this, it is a time-consuming assessment to complete (up to 45 minutes per person) and in reality the majority of participants refused to complete it.' (p59) Most of the reports did not include such candid reflections on the merits or otherwise of the measures selected for use.Neither did many include reflection on the data collection processes or the experiences of research workers or research participants.Process evaluations tended to focus on the process of implementing the studied intervention or recruiting participants, not data collection per se.However, five of the 17 reports did include some form of process evaluation or embedded study that touched on the process of data collection from people with dementia [45,51,55,56,59] EVIDEM-C [56] included interviews with carer participants about their experiences of data collection, but people with dementia were not interviewed.Allan et al. [51] interviewed participants with dementia, carers and research staff.The most common concern gleaned from their combined responses was that the baseline and follow-up assessments took too long to complete.O'Brien et al. [59] collected feedback from people with dementia as well as carers and clinicians on the measures to be included in their assessment toolkit and reported that '… patients and carers highlighted some issues with question wording' (p39).They noted 'tensions between research paradigms' (p44), in particular the value ascribed to validated questions versus qualitative feedback from participants, but offered no further details.
With reference to field notes, Gridley et al. [55] identified a number of challenges inherent in collecting data from participants with dementia including the capacity and frailty of the participants; the context within which data collection took place (e.g.care homes where staff had other priorities); and the geographic location of the research settings, compared to that of the research team (given that data collection with people with dementia can be time consuming and require multiple visits).However, they also identified the closed-question format of the standardised measures as a key reason for low response rates.Clinical trial assistants (CTAs) interviewed for the Allen et al. [51] process evaluation reported concerns that the wording of some measures was difficult for participants with more advanced dementia to understand, for example because they contained double negatives.They also felt that some participants with dementia found the questions ambiguous and needed further explanation, which they had been trained not to give as this could impair the standardisation of the measure.The authors concluded that research workers like the CTAs require better training in the administration of standardised measures to ensure a consistent approach.
Clarkson et al. [45] had the most to say about the data collection context and process, and the influence of these on the data collected, producing an accompanying paper dedicated to reflecting on the research encounter.This paper was based on findings from an embedded qualitative study in which researchers audio-recorded the data collection process, revealing the dialogue surrounding the answers given to closed questions [14].They noted that even people in the early stages of dementia 'struggled with the structured and standardised nature of the research interviews, finding them a linguistic and cognitive challenge' ( [45], p15).They also noted the work that researchers had to undertake to determine whose perspective was being addressed, when family carers were present during data collection sessions with people with dementia.

Emotional distress
The potential for standardised data collection to cause emotional distress in participants with dementia was explicitly identified as a risk in two of the reports [45,55] and implied in a third [51].While most of the reports made no mention of question content, Gridley et al. [55] noted the potential impact of sensitive or negative questions on participant wellbeing: '…we found that, for example, asking people in quick succession whether they had lately felt sad (question 7), lonely (question 8) and then distressed (question 9) could trigger sadness.On one occasion (plus on two occasions in hospital wards) DEMQOL was abandoned specifically for this reason.' ( [60], p69) Clarkson et al. [45] similarly noted that the measures they used addressed sensitive topics 'that could be distressing for people with dementia and their carers and difficult for interviewers to manage.' (p15).Their accompanying paper identified that some standardised questions could be 'very direct in probing potentially emotionally difficult aspects of life, particularly in the context of older age and deteriorating cognition' ([14], p2742).
Interviews with carers for the process evaluation by Gridley et al. also suggested that some people could find the experience of being questioned worrying in itself.Again, this concurs with the account of Abendstern et al. [14] suggesting that, for some, the structured interview as a whole appeared to cause anxiety: 'This was indicated in several ways including misunderstanding questions and showing uncertainty about how to reply, giving answers that they seemed to think the interviewer wanted, conveying feeling pressured to say the right thing, and forgetting things during the memory 'test' ….Some participants expressed distress at the prospect of the interview itself, commenting that they were unsure about what to expect…' (p2741) The most common concern of those interviewed in the Allan et al. [51] process evaluation was the length of time it took to complete the baseline and follow-up assessments.However, through their illustration of this challenge it is evident that the data collection process in this study was also associated with, or may even have caused, emotional distress in some participants: '…for the patients, it was a bit too much when you're sat in the house.We only had, like, 90 minutes but I couldn't do the first one in less than 2 hours because he kept getting upset and crying, it was very difficult.' Professional 145, CTA (interview) ( [51], p93) Such issues are generally not reported (or even recorded) in trials involving people with dementia.Together with the practice noted above of pausing data collection part way through (either if requested by participants or deemed appropriate by researchers) the issues highlighted by these three reports raise questions about participants' experiences during data collection and the degree to which not only fatigue, but also emotional distress, may be features of the data collection process worthy of further investigation.

Discussion
In this paper we presented a narrative synthesis of the reported use of standardised, self-report measures with people with dementia in 22 NIHR funded studies selected systematically from the NIHR Journals Library website.Response rates (where these could be ascertained) varied considerably and appeared to be related to dementia severity and place of residence, whilst measure completeness and patterns of item nonresponse were rarely reported.Overall, we found little reported information about the process of data collection from people with dementia (over and above basic setting and mode) or reasons for missing data.There was also very little information about the experiences of participants with dementia or those administering the measures.However, from the few instances where experiences were reported it seems that there may be risks to participants' well-being associated with both the content of measures and context of data collection that are worthy of further consideration.
Despite some discrepancies in reporting, it was clear from the review that measures were not always completed in full at all time points and that some measures were not completed at all by some participants, even those still included in study samples.Such gaps are common in research; 100% response rates are rare [62] and missing data has been identified as a particular problem in research on ageing [63].Some of the response rates reported in this review, however, seem to be considerably lower than would be expected for the general population, less than 50% in several cases, whilst in other cases they were close to 100%.Some of the apparent variation in response rates may be artefacts of reporting (for example, some studies amended sample sizes at follow-up in response to withdrawal, whilst others calculated response rates at follow-up using original sample sizes).However, it is clear that some studies faced real challenges in their attempts to obtain self-reported data from participants with dementia, which other studies appeared to avoid.The lack of detail on measure administration and participant experience means it was not always possible to determine the reasons underpinning these differences.
Studies that included people with more severe dementia tended to report more problems obtaining consistent response rates, supporting previous research where cognitive impairment has been shown to predict item nonresponse [15] or recourse to dichotomous (yes/no) answers [40].Those which only included participants with milder cognitive impairment reported fewer problems and, where response rates were clearly reported, these were generally over 90% at baseline.In contrast, studies which included participants with all stages of dementia (i.e.including people with severe dementia) reported response rates ranging from 0 to 100%, with many under 75% or not reported.One approach used by some teams to minimise missing data was to apply tight eligibility criteria.However, while high response rates are desirable from a statistical perspective, restricting the eligibility criteria creates a trade off with generalisability, as the outcomes and perspectives of a group of people who could potentially be affected by the intervention under evaluation may not be included in the results [64,65].
Little attention was paid in most reports to the potential risk to participants of emotional distress, despite previous flagging of this in the published literature, particularly in the context of qualitative research: 'All too often the person with dementia can be left with the feeling of not being able to do, not being able to remember or not reaching the right score, so they can feel excluded and a failure.' ( [66], p817) Such risks have also been noted in quantitative data collection [26,[67][68][69] and there is some indication that standardised tests of cognitive impairment can be particularly problematic [21,39,70].The bulk of literature highlighting the tension between the requirements of standardisation and the wellbeing of participants has focussed on data collection in clinical settings [71][72][73].However, research participants may also experience feelings of anxiety and people with dementia may be particularly susceptible to emotional distress or agitation brought on or exacerbated by the research encounter [74][75][76].
Failing to understand the reasons behind missing data has implications for future successful data collection and appraisal of the appropriateness of measures.For example, if data are consistently missing for an item or measure because participants find it distressing to answer, the implications and possible remedies will differ from scenarios where items are skipped or measures dropped because they were found to be too cognitively challenging.The issues related to entire measures going unanswered by particular participants may also be different from the possible reasons behind individual missing items [62].Patterns in item non-response may reflect problems with particular item wording specific to the communication and cognitive function of people with dementia which remain unaddressed without systematic examination [15].Alternatively, missing data may be unrelated to the content or structure of the measures, but instead be the result of contextual factors such as care home practices: perhaps participants were not available at the allotted interview times [77], or carers were not available to provide support on the day.Certainly, in this review, studies attempting to collect data from participants with dementia residing in care homes or hospital wards appeared to achieve lower response rates than those collecting data from people residing in the community, perhaps reflecting known barriers to the undertaking of research in residential settings [78,79].
A common solution proposed to the challenges of collecting research data directly from people with dementia is to use data from proxy measures alongside, or even instead of, self-reported data.However, in addition to the ethical issues of reliance on another person's views in place of the person with dementia's [69], some of the reports in this review flagged methodological issues inherent in this approach, such as the various relationships of proxies to participants [49], and the tendency for proxies to rate quality of life lower than people with dementia do themselves [57,58].This fits with the findings of multiple previous studies [30,80,81] and calls into question the ability of proxy measures to validly represent the views of people with dementia.At the very least, proxies may be reporting something conceptually different from the thing people with dementia themselves are reporting when asked about their 'quality of life' [82,83].
An alternative solution would be to design methods or select measures more likely to be comprehensible, manageable and meaningful to people with dementia [84], that is, measures that are a better 'fit' for the people affected by the intervention so that they can answer for themselves [85].Dementia specific quality of life measures like DEMQOL and QOL-AD were designed to do this, but the results of this review call into question the appropriateness of using even current dementia specific measures without additional support for some participants with dementia.Indeed, DEMQOL was only validated using data from people with mild to moderate dementia (data from people with an MMSE score of less than 10 were excluded from the analysis [18], and whilst it is commonly quoted that QOL-AD is suitable for use with people with an MMSE score as low as 3 (based on a 2003 study by Thorgrimsen et al. [86]), Kinderman et al. [57] struggled to use this measure with people with severe dementia in residential settings: 'Although it has been suggested that the QOL-AD can be usefully completed with some people with a MMSE score of as low as 3 (although it was originally suggested to be valid for use with people with MMSE scores of > 10), it quickly became obvious that the majority of people living with dementia in the care homes and wards visited were unable to complete the measure, even with assistance from skilled clinicians.' (p53) Without more detailed descriptions of what happens when researchers attempt to administer measures, and the individual items within those measures, it is hard to ascertain exactly which elements of measures, or research context, may be problematic and require attention.
Accounts from clinical settings have highlighted the, often marked, difference between the standardised conditions envisaged by those who design measures [87] and the realities of measure administration in practice.As Krohne explains: '…test administrators must deal with interruptions, such as test-takers falling asleep, being in pain, not understanding the question, or consciously choosing not to respond to the question.' ( [88], p29).
Conventions in standardised interviewing [89,90] along with the specific instructions for some measures (such as DEMQOL [18]) preclude the giving of support or explanations that are not in the script, even for participants with cognitive impairment.Yet there is evidence that standardisation in practice is difficult to achieve, even with participants with no cognitive impairment [91][92][93][94].As Gobo, Giampietro, and Mauceri note [94] the standardised interview is 'an interaction that takes place in a social situation' (pXVII).Responses to accounts of researchers having difficulties adhering to standardisation tend to take the form of calls for better training (such as that in Allan et al. [51], but whilst training researchers to apply greater consistency in handling participants' queries may reduce the chances of interviewer bias, this would not necessarily address participants' anxieties, or any underlying problems with the measures themselves.
It is necessary to find a way through such dilemmas, as simply excluding specific groups from research participation because of their perceived vulnerability is no longer considered acceptable [95] and is certainly unacceptable to an increasingly politically conscious population of people living with dementia.Several key documents have been published in collaboration with people with dementia in recent years setting out, amongst other things, their right to be involved in research that concerns them [5,6,96].Dementia care theorists advocate a personalised approach to working with people with dementia [97,98] and a growing literature from the qualitative traditions have argued for greater flexibility in data collection, as Keady notes: [Participants with dementia often have] difficulties with linguistic, behavioural and cognitive functioning.Researchers therefore need to be creative and adapt their methods of data collection in order to address the individual needs of someone who is living with dementia.' ( [13], p2) The principles of creativity and adaptability to the needs of the individual do not, however, sit well with the fundamentals of quantitative measurement, which rely on standardisation: essentially, inflexibility.This raises the question of whether quantitative data collection can feasibly be reconciled with the principles of best practice in dementia research.Evans et al. [99] looked at the relationship between reported quality of life scores and interviewer continuity and concluded that having a familiar person visit to collect follow-up data might influence results.The suggestion here is of a conflict between more person-centred approaches and data integrity, as opportunities to build rapport and put participants at ease might also lead to interviewer bias.However, those findings were based on exploratory secondary analysis of a completed study and the authors noted that 'characteristics, such as age, training, experience, warmth and ability to establish rapport, were not taken into account (given the lack of data)' ([99], p7).More attention must be paid to these factors, and the experiences of the people involved, to fully understand what influences scores and/or leads to missing data.
If compromises between standardisation and personalisation must be made, one solution proposed by Phillipson et al. [100] is to offer incremental levels of support including physical and emotional support, in addition to 'easy read' documentation, where this might facilitate the inclusion of people with a greater degree of impairment.Whilst this involves more flexibility than is usually permitted in standardised data collection, Phillipson et al. demonstrate that the provision of such tailored support enabled them to collect data from people who would not have been able to participate, or would have had large quantities of missing data, had they not been supported to respond, and their findings were richer because of this.However, this study only used one measure, the ASCOT, whereas some of the studies included in this review used multiple (up to 10 different) self-report measures with each person at each time point.It is likely that there would be a trade-off between the amount of tailoring of support practically achievable in a study and the number of measures used, with implications for the time and other resources required.It remains to be seen how applicable such an approach could be to a large trial or cohort study with multiple measures.

Limitations
This review was conducted as part of a doctoral research project and as such is based primarily on the independent work of one researcher.However, two supervisors (both experienced senior health and social care researchers) were closely involved throughout and are co-authors on the paper.Moreover, the findings have been discussed more widely with colleagues working in the health and social care research field, including with those specialising in dementia research, with feedback integrated into our interpretation of findings.Nevertheless, it is recognised that working relatively independently, whilst necessary for doctoral studies, is a limitation in any review and the results should be read with this in mind.
The review only covered research published in a single, UK database and there may be learning from other types of research and research reporting not covered here.However, the NIHR is Britain's largest funder of clinical, public health and social care research, and its Journals Library contains comprehensive, open access accounts of final, peer reviewed reports including methods and a full description of the results [101].As such, the review offers a useful insight into the reporting of high status government funded dementia research and poses questions of relevance to the wider field.

Fig. 1
Fig. 1 PRISMA flow diagram of assessment, exclusion and inclusion

Table 1
Self-report measures used with people with dementia and response rates (studies arranged by dementia severity)

Table 1 (continued) Citation and included study N with dementia, severity Self-report measures, primary outcome Response rates for self-report measures by time point
[48]Orgeta V, Leung P, Yates L, Kang S, Hoare Z, Henderson C, et al.Individual cognitive stimulation therapy for dementia: A clinical effectiveness and cost-effectiveness pragmatic, multicentre, randomised controlled trial.Health [49] Orrell M, Hoe J, Charlesworth G, Russell I, Challis D, Moniz-Cook E, et al.Support at Home: Interventions to Enhance Life in Dementia (SHIELD) -evidence, development and evaluation of complex interventions.Program Grants Appl Res.2017;5(5):1-184 Included study: RCT of Maintenance Cognitive Stimulation Therapy (MCST) N = 236 participants were randomised to the MCST or usual care group (half recruited from care homes and half recruited from community settings) Mild to moderate Inclusion criteria specified that participants had a CDR of between 0.5 to 2. 'Most of the sample had moderate dementia, with a mean MMSE score of 16.8 (SD 5.5) and a mean ADAS-Cog score of 34.3 (SD 12.9)' (pg30)

Table 1 (continued) Citation and included study N with dementia, severity Self-report measures, primary outcome Response rates for self-report measures by time point
[49]Orrell M, Hoe J, Charlesworth G, Russell I, Challis D, Moniz-Cook E, et al.Support at Home: Interventions to Enhance Life in Dementia (SHIELD) -evidence, development and evaluation of complex interventions.Program Grants Appl Res.2017;5(5):1-184 Included study: Maintenance Cognitive Stimulation Therapy implementation study (observational) N = 89 Mild to moderate Inclusion criteria specified that participants had a CDR of between 0.5 to 2. [50] Woods R, Bruce E, Edwards R, Elvish R, Hoare Z, Hounsome B, et al.REMCARE: Reminiscence groups for people with dementia and their family caregivers -Effectiveness and costeffectiveness pragmatic multicentre randomised trial.Health Technol Assess.2012;16(48):v-116.Included study: RCT of group reminiscence for people with dementia and carers N = 487 Mild to moderate Inclusion criteria specified that participants were in mild to moderate stage of dementia based on CDR Self-reported measures: QOL-AD EQ-5D QCPR One primary outcome measure was selfreported QOL-AD.The other was caregivers' mental health 350 dyads 'completed the study' .Response rates not clearly reported by time point.Studies collecting data from people with all stages of dementia: [51] Allan LM, Wheatley A, Smith A, Flynn E, Homer T, Robalino S, et al.An intervention to improve outcomes of falls in dementia: The DIFRID mixed-methods feasibility study.Health Technol Assess.2019;23(59):1-20 Included study: Feasibility study of a falls intervention

Table 1 (continued) Citation and included study N with dementia, severity Self-report measures, primary outcome Response rates for self-report measures by time point
[52] Banerjee S, Hellier J, Romeo R, Dewey M, Knapp M, Ballard C, et al.Study of the use of antidepressants for depression in dementia: The HTA-SADD trial-A multicentre, randomised, double-blind, placebo-controlled trial of the clinical effectiveness and cost-effectiveness of sertraline and mirtazapine.Health Technol Assess.2013;17(7):1-43.Included study: Multicentre RCT of two antidepressants for people with dementia N = 326 All stages eligible (one participant excluded because they had severe dementia SMMSE < 8) Mean (SD) SMMSE at baseline for the 3 groups: 18.2 (7.4); 18.5 (6.7), and 17.6 (6.0) Self-reported measures: [53] Bowen M, Edgar DF, Hancock B, Haque S, Shah R, Buchanan S, et al.The Prevalence of Visual Impairment in People with Dementia (the PrOVIDe study): a cross-